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•  Idealize:  define  an  ideal  future  state 

•  Diagnose:  identify  the  key  issues 

•  Explore:  brainstorm  creative  strategies  for  addressing  the 
issues  identified 

•  Assess  and  analyze:  evaluate  potential  strategies 

•  Story:  wrap  the  resulting  solution  into  a  story  that  wins  the 
interest  and  support  of  the  key  stakeholders  needed  to 
bring  the  idea  to  fruition. 1 


1.  Kaihan  Krippendorff,  Unlocking  Innovation:  Out-think  Your  Competitions.  www.kaihan.net/Unlocking_innovation.pdf 
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WG  3  Schedule 
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•  Tuesday 

-  1010-1130 

-  1300-1630 

Session  1  -  Introduction,  Approach,  Objectives 

Session  2  -  System  of  Systems,  Measurement, 

C2  Framework  (Conceptual  Model), 

OR  (&  Other)  Methods 

•  Wednesday 
-  0845-1200 

Session  3  -  C2  Framework,  OR  Methods 

MOOs,  MOEs,  MOPs  Development 

Case  Study 

-  1300-1630 

•  Thursday 

-  0800-1200 

_  1420-1450 

Session  4  -  Findings,  Recommendations,  Conclusions 

Session  5  -  Outbrief  Preparation 

WG  3  Outbrief 

Clyde  S.  Smithson  III  UNCLASSIFIED 
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•  Because  networked  C2  systems  are  actually  “systems  of 
systems”  this  working  group  will  examine  the  difficulties 
of  applying  operations  research  techniques  to  them  in  this 
context.  Networked  systems  exhibit  behavior  that  is 
complex  and  defies  the  usual  techniques  of  measuring 
effectiveness  at  discrete  points.  This  working  group  will 
explore  the  correct  approaches  for  measuring  and  assessing 
network  behaviors  and  the  effectiveness  of  the  network. 


Clyde  S.  Smithson  III 
23-26  January,  2012 


UNCLASSIFIED 

www.WIORS.org  6 


WG3:  Operations  Analysis  for  SoS  within  a  Networked  C2  Context 


Military  Operations  Research  Society 


Objectives 
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•  Objective  1 :  Understand  the  impact  of  the  application  of 
traditional  operational  research  techniques  to  networked 
C2  systems. 

•  Objective  2:  Develop  inputs  to  the  C2  Metrics  Framework 
for  networked  C2  systems  and  “systems  of  systems”  to 
measure  and  assess  network  behaviors. 

•  Objective  3:  Identify  and  categorize  families  of  C2 
measures  of  effectiveness  useful  for  networked  C2 
systems. 


•  Task  1:  Challenge  these  objectives 
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C2  Network  Effectiveness 

Framework 
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1.  Mission  and  Context  definition 

2.  What  is  the  minimum  data  content  needed  to  produce  desired  behavior?  What  is 
the  optimal  (locally  optimal,  local  sub-optimization  to  achieve  SOS  goals)  data 
content?  Might  be  akin  to  fidelity  (scale  and  scope)  measures  to  be  defined  for  the 
network. 

3.  Cost  measures  including  cost  and  time  to  implement  the  solution  (for  example,  a 
basic  rule-of-thumb  I  use  for  development,  integration,  fielding  of  a  new  TADIL-J 
message  is  on  the  order  of  $1B).  Acceptable  risk.  How  these  factors  would  form  a 
tradespace.  Relating  effectiveness  measures  back  to  WG  -  1  &  2. 

4.  Includes  concepts  such  as: 

-  System/network  boundaries  -  explicit  &  derived  assumptions,  entities  &  interactions 
(nodes  &  links) 

-  System/network  behaviors  -  entities,  events,  states,  functions,  time 

-  System/network  forms  -  architecture,  bandwidth,  latency,  error,  etc. 

5.  Other  topics 

-  Network  permeability,  secure  networks;  relate  to  risk 

-  Emergent  Behavior  (at  multiple  levels) 
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Paradigm 
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•  Commander’s  Intent 

•  Mission  Outcome/Success 

•  C2  Network  Layers 

-  Physical 

-  Information 

-  Cognitive 
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1.  Quality  of  Information  (QOI),  Quality  of  Service  (QoS)  metrics  - 
that  describe  the  impact  to  the  behavior  of  the  SOS;  especially  in 
the  context  of  outcomes  I  think  taking  a  look  at  something  like  on 
OSI  7-layer  model  as  a  way  to  categorize  metrics.  My  sense  is  that 
most  of  the  discussion  would  be  in  the  Host  Layer  (Application, 
Presentation,  Session,  Transport)  to  describe  behavior;  however, 
the  impact  of  lower  layers  will  be  important  as  I  think  there  may 
be  metrics  associated  with,  for  example,  bandwidth  (which  may  in 
turn  drive  media  -  fiber,  RF,  IR,  EO,  etc.)  as  low  as  at  the  Physical 
Layer. 

2.  Cloud  Computing  Paradigm  (Apps/Mission,  Methods/Tools, 
Transport/Network) 

3.  A  method  to  relate  effectiveness  metrics  back  to  cost  (and  possibly 
schedule)  and  risk. 
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•  Tutorials 

-  Day  1  Tutorials  ( TBD ) 

-  Metrics  101  [Condensed  Edition]  &  201-  R.  W.  Eberth 

•  Reading 

-  NATO  Code  of  Best  Practices  for  C2  Assessment 

-  Command  &  Control:  The  Sociotechnical  Perspective 

-  Formulating  Measures  of  Effectiveness  -  N.  Sproles 

-  Coming  to  Grips  with  Measures  of  Effectiveness  -  N.  Sproles 

-  The  Difficult  Problem  of  Establishing  Measures  of  Effectiveness  for 
Command  and  Control:  A  Systems  Engineering  Perspective  -  N. 
Sproles 

-  An  Approach  to  Simulation  Effectiveness  -  D.  Goncalves  (this  last  is 
provided  as  a  thought  exercise  and  example;  can  a  similar  technique  be  applied 
to  describing  C2  Network  Effectiveness?) 
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Measures 
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•  Measures  of  Outcome 

-  Commander’s  Intent 

-  Mission  Success 

•  Measures  of  Effectiveness 

-  Effectiveness,  Efficiency 

-  Quality  of  Information 

-  Cost,  Schedule,  Risk 

•  Measures  of  Performance 

-  Quality  of  Service 

-  Network  Metrics 
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Systems  of  Systems  Definition 
(DoD  SE  Guide  for  SoS) 

UNCLASSIFIED 

•  System 

-  A  functionally,  physically,  and/or  behaviorally  related  group  of 
regularly  interacting  or  interdependent  elements;  that  group  of 
elements  forming  a  unified  whole  [JP  1-02  &  JP  3-0]. 

•  Capability 

-  A  capability  is  the  ability  to  achieve  a  desired  effect  under  specified 
standards  and  conditions  through  combinations  of  ways  and  means  to 
perform  a  set  of  tasks  [CJCS,  2007(2)]. 


•  System  of  Systems 


-  An  SoS  is  defined  as  a  set  or  arrangement  of  systems  that  results 
when  independent  and  useful  systems  are  integrated  into  a  larger 
system  that  delivers  unique  capabilities  [DoD,  2004(1)].  Both 
individual  systems  and  SoS  conform  to  the  accepted  definition  of  a 
system  in  that  each  consists  of  parts ,  relationships ,  and  a  whole  that  is 
greater  than  the  sum  of  the  parts;  however,  although  an  SoS  is  a  system, 
_ not  all  systems  are  SoS. _ 

Clyde  S.  Smithson  III  UNCLASSIFIED 
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Describing  Systems,  Systems  of 
Systems,  Complex  Systems 


The  nature  of  large  complex  system  development  is  contrasted  with  that  for  a  traditional  system.  In 
the  traditional  case  a  legacy  system  capability  may  be  able  to  be  swapped  out  entirely  by  a  new 
system.  This  case  tends  to  prioritize  in  order  of  technical,  operational,  economic,  and  political 
(TOEP)  merits.  For  a  SoS  it  is  much  more  complex  to  introduce  new/improved  capability  without 
affecting  overall  capability  due  to  the  number  of  systems,  complex  interactions  between  systems, 
their  unique  capabilities,  and  the  different  development/acquisition  organizations.  Much  more 
must  be  balanced  and  negotiated  across  the  SoS  so  priorities  tend  to  align  to  political,  operational, 
economic,  and  technical  (POET)  merits.  One  can  argue  the  relative  positions  of  operational  and 
economic  in  this  case. 


Because  of  the  complexity  of  the  socio-technical  system  involved  it  is  critical  to  first  frame  the 
effectiveness  referent  and  the  problem  context,  especially  across  the  various  social  entities  to 
determine  driving  factors  for  each  and,  hopefully,  establish  common  problem  characteristics  across 
multiple  organizations.  These  common  characteristics  will  serve  as  the  basis  from  which  to 
formulate  measures  of  merit  for  the  SoS  Effectiveness  &  Performance  assessment.  The  measures  of 
merit  should  be  of  both  the  qualitative  and  quantitative  type,  and  be  able  to  tolerate  a  certain  level 
of  ambiguity  across  the  SoS. 

The  POET  nature  of  the  SoS  indicates  that  the  development,  testing  and  integration,  and 
assessment  of  the  system  cannot  strictly  be  addressed  through  traditional  methods  alone,  such  as 
operations  research,  systems  analysis,  or  traditional  system  engineering.  Although  these  form  a 
backbone  for  addressing  the  SoS  problem,  especially  when  viewing  the  individual  systems/sub¬ 
systems/components  forming  the  SoS.  The  wide  Stakeholder  base,  multi-mission  nature  of  some 
parts  of  the  SoS,  and  the  open  nature  of  the  of  most  C2  systems(with  vague  and  permeable  system 

boundaries)  means  that  there  may  be  no  single  simple  view  to  which  the  system  can  be  reduced. 

Clyde  S.  Smithson  III  UNCLASSIFIED 
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•  In  DoD  and  elsewhere,  SoS  can  take  different  forms.  Based  on  a 
recognized  taxonomy  of  SoS,  there  are  four  types  of  SoS  which 
are  found  in  the  DoD  today  [Maier,1998;  Dahmann,  2008]. 

—  Virtual.  Virtual  SoS  lack  a  central  management  authority  and  a  centrally  agreed  upon  purpose 
for  the  system -of-systems.  Large-scale  behavior  emerges — and  may  be  desirable — but  this 
type  of  SoS  must  rely  upon  relatively  invisible  mechanisms  to  maintain  it. 

—  Collaborative.  In  collaborative  SoS  the  component  systems  interact  more  or  less  voluntarily 
to  fulfill  agreed  upon  central  purposes.  The  Internet  is  a  collaborative  system.  The  Internet 
Engineering  Task  Force  works  out  standards  but  has  no  power  to  enforce  them.  The  central 
players  collectively  decide  how  to  provide  or  deny  service,  thereby  providing  some  means  of 
enforcing  and  maintaining  standards. 

—  Acknowledged.  Acknowledged  SoS  have  recognized  objectives,  a  designated  manager,  and 
resources  for  the  SoS;  however,  the  constituent  systems  retain  their  independent  ownership, 
objectives,  funding,  and  development  and  sustainment  approaches.  Changes  in  the  systems 
are  based  on  collaboration  between  the  SoS  and  the  system. 

—  Directed.  Directed  SoS  are  those  in  which  the  integrated  system -of-systems  is  built  and 
managed  to  fulfill  specific  purposes.  It  is  centrally  managed  during  long-term  operation  to 
continue  to  fulfill  those  purposes  as  well  as  any  new  ones  the  system  owners  might  wish  to 
address.  The  component  systems  maintain  an  ability  to  operate  independently,  but  their 
normal  operational  mode  is  subordinated  to  the  central  managed  purpose. 

Maier,  M.  (1998);  "Architecting  Principles  for  Systems-of-Systems";  Systems  Engineering,  Vol.  1,  No.  4  (pp  267-284). 

Dahmann,  Judith  and  Kristen  Baldwin,  (2008),  “Understanding  the  Current  State  of  US  Defense  Systems  of  Systems  and  the  Implications  for  Systems 
Engineering”,  Montreal,  Canada:  IEEE  Systems  Conference,  7-10  April. 
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Aspect  of 
Environment 

Acknowledged  SoS* 

Virtual  SoS 

Collaborative  SoS 

Directed  SoS 

Management  &  Oversight 

Stakeholder 

Involvement 

Stakeholders  at  both  system 
level  and  SoS  levels  (including 
the  system  owners),  with 
competing  interests  and 
priorities;  in  some  cases,  the 
system  stakeholder  has  no 
vested  interest  in  the  SoS;  all 
stakeholders  may  not  be 
recognized. 

A  virtual  SoS  has  no  centrally 
established  purposes  but  rather  the 
puipose  expresses  itself  as  the 
collective  actions  of  the  individual 
systems. 

Stakeholders  negotiate  among 
themselves  to  establish  a  common 
puipose.  The  SoS  is  built  to  this 
purpose  and  the  individual  systems 
negotiate  among  themselves  to 
determine  which  part  of  this 
responsibility  each  fulfills.  Central 
players  often  establish  the  ground  rules 
by  which  other  players  participate. 

A  central  SoS  authority  usually 
establishes  the  purpose  to  be 
achieved  by  the  SoS.  The  SoS 
is  built  to  this  purpose  and  the 
individual  systems  are 
generally  directed  by  the 
central  authority. 

Governance 

Added  levels  of  complexity 
due  to  management  and 
funding  for  both  the  SoS  and 
individual  systems;  SoS  does 
not  have  authority  over  all  the 
systems. 

No  central  body  controls  the  purpose 
or  management  of  the  SoS  or 
individual  systems.  Governance  may 
emerge  from  politics  or  policies 
agreed  to  by  stakeholders  but  none  is 
compelled  to  comply. 

In  collaborative  SoS  there  is  no  central 
authority  with  the  power  to  enforce  a 
particular  SoS  purpose.  A  central 
authority  may  establish  purposes, 
standards,  etc.,  which  are  usually 
complied  with,  but  does  not  have 
authority  to  enforce  them. 

Individual  systems  are 
governed  by  membership  to  a 
common  SoS  command 
structure  which  usually 
includes  a  central  governing 
authority. 

Operational  Environment 

Operational 

Focus 

Called  upon  to  meet  a  set  of 
operational  objectives  using 
systems  whose  objectives  may 
or  may  not  align  with  the  SoS 
objectives. 

Individual  systems  are  operated 
independently.  Operation  of  the  SoS 
is  complex  because  there  is  no 
centrally  directed/controlled  puipose. 
Participation  by  systems  is  voluntary 
and  they  often  have  conflicting 
purposes  which  they  will  try  to  attain 
simultaneously  with  other  systems. 

Collaborative  SoS  differs  from  directed 
SoS  in  that  a  central  authority  is  not 
able  to  enforce  particular  operation  of 
the  system.  Systems  collaborate  of  their 
own  will  to  achieve  a  central  purpose; 
however,  from  time  to  time  SoS 
operational  needs  are  subjugated  to  the 
needs  of  a  particular  system. 

The  systems  are  connected  by 
command  and  control 
structures.  The  SoS  directs  the 
operation  of  individual  systems 
to  achieve  the  SoS  purpose  (a 
centralized  control  authority). 
Systems  are  usually  allowed 
operational  independence  to 
deal  with  local  situations. 

Clyde  S.  Smithson  III  *Table  adapted  &  Acknowledged  SoS  definitions  from  DoD  SE  Guide  for  SoS;  oU  NC  LAS  SI  Fi  ED.  h  o  r. 
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Aspect  of 
Environment 

Acknowledged  SoS* 

Virtual  SoS 

Collaborative  SoS 

Directed  SoS 

Implementation 

Acquisition 

Added  complexity  due  to 
multiple  system  lifecycles 
across  acquisition  programs, 
involving  legacy  systems, 
systems  under  development, 
new  developments,  and 
technology  insertion;  Typically 
have  stated  capability 
objectives  upfront  which  may 
need  to  be  translated  into 
formal  requirements. 

Component  systems  are  acquired 
independently  without  regard  to  other 
systems,  except  in  the  context  that 
another  system  may  perform  a 
beneficial  function  for  that  system 
(usually  at  little  or  no  cost)  and 
dependably. 

Systems  negotiate  among  themselves  to 
determine  how  SoS  objectives  are  to  be 
met  and  which  system  is  to  provide 
which  SoS  capability.  Agreements  are 
made  between  central  players  to  form  a 
common  acquisition  strategy.  This  can 
be  seen  as  negotiated  “political” 
objective  as  opposed  to  direction  by 
central  authority. 

Individual  systems  are 
acquired  through  different 
program  offices  and  operated 
separately;  however,  there  is  a 
central  authority  directing, 
coordinating,  and  balancing  the 
various  program  offices. 
Systems  may  be  custom  built 
to  meet  the  needs  of  the  SoS. 

Test& 

Evaluation 

Testing  is  more  challenging 
due  to  the  difficulty  of 
synchronizing  across  multiple 
systems’  life  cycles;  given  the 
complexity  of  all  the  moving 
parts  and  potential  for 
unintended  consequences. 

SoS  testing  generally  occurs  on  an  ad 
hoc  basis.  Individual  systems  test 
themselves.  Testing  at  the  SoS  level 
is  confined  to  aspects  of  the  SoS  at 
that  level  that  affect  the  function  and 
purpose  of  individual  systems.  In 
other  words  a  system  only  tests  what 
is  important  to  itself  at  the  SoS  level, 
if  any  SoS  testing  is  conducted  at  all. 

SoS  testing  is  established  by 
coordination  and  negotiation  between 
the  central  SoS  players.  Testing  tends 
to  change  over  time  as  the  SoS  purpose 
evolves.  For  a  directed  SoS  the  testing 
tends  to  be  directed  from  top  down 
whereas  for  virtual  SoS  it  springs  up 
organically.  T&E  for  a  collaborative 
system  comes  from  a  middle  ground  in 
which  the  central  players  establish  goals 
that  are  tested  by  the  entire  SoS. 

Testing  occurs  at  multiple 
levels  but  is  directed  from  the 
SoS  level  At  the  SoS  level 
testing  is  directed  to  evaluate 
the  central  purpose  of  the  SoS. 
Testing  may  occur  with  the 
entire  SoS  or  portions  of  it. 
Additionally,  testing  occurs  at 
the  system  level  to  establish 
that  the  system  meets  its 
individual  requirements, 
including  those  supporting  the 
system  purpose. 

*  Tabic  adapted  &  Acknowledged  SoS  definitions  from  DoD  SE  Guide  for  SoS;  others  defined  by  this  author. 
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Aspect  of 
Environment 

Acknowledged  SoS* 

Virtual  SoS 

Collaborative  SoS 

Directed  SoS 

Engineering  &  Design  Considerations 

Boundaries 

and 

Interfaces 

Focus  on  identifying  the 
systems  that  contribute  to  the 
SoS  objectives  and  enabling 
the  flow  of  data,  control  and 
functionality  across  the  SoS 
while  balancing  needs  of  the 
systems. 

Boundaries  and  interfaces  evolve 
through  adaptation  and  survival  - 
successful  standards  live  and  are 
extended  upon  while  others  die  out. 
Forces  other  than  the  technical  merits 
of  these  may  determine  survival  (e.g., 
VHS  vs.  Betamax).  Systems  choose 
to  use  or  not  use  these  at  their  own 
discretion.  A  standard  may  be  created 
by  an  individual  system,  and  then  be 
adopted  by  others. 

Certain  systems  rise  to  be  central 
players  at  the  SoS  level.  These  systems 
usually  reach  agreement  on  what  the 
interface  standards  are  and  what 
services  to  provide.  They  usually  create 
common  standards  for  use  by  the  entire 
SoS  but  do  not  enforce  them  (except  by 
operationally  excluding  other  systems 
that  do  not  conform). 

Interfaces  are  seen  as  a  key 
integrating  factor  for  the  SoS. 

A  central  authority  establishes 
the  interface  requirements, 
with  input  from  the  component 
systems.  Similarly,  the  central 
authority  establishes  the 
boundaries  between  systems. 

Performance 
&  Behavior 

Performance  across  the  SoS 
that  satisfies  SoS  user 
capability  needs  while 
balancing  needs  of  the 
systems. 

The  performance  of  the  SoS  is  not 
directed,  but  rather  is  an  emergent 
behavior.  There  are  no  established 

SoS  performance  requirements. 
Individual  systems  optimize  to 
perform  best  for  their  own  ends  (i.e., 
best  ROI  at  the  system  level)  and  SoS 
performance  derives  from  that. 

Like  the  virtual  SoS,  there  are  no 
minimum  SoS  performance 
requirements  enforced  by  a  central 
authority.  Rather,  the  constituent 
systems  agree  to  a  set  of  mutual 
performance  goals  and  behaviors  which 
evolve  over  time.  Individual  systems 
may  choose  to  sub  optimize  to  benefit 
the  SoS. 

All  constituent  systems  must 
met  minimum  performance 
requirements  to  satisfy  SoS 
capability  requirements. 
Individual  systems  may  be 
operated  sub  optimally  to  meet 
SoS  performance  requirement. 
Generally,  individual  system 
performance  is  secondary  to 

SoS  performance. 

*  Tabic  adapted  &  Acknowledged  SoS  definitions  from  DoD  SE  Guide  for  SoS;  others  defined  by  this  author. 
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Characteristic 

Description 

Operational 

Independence 

The  constituent  Systems  of  the  SoS  are  often  able  to,  and  do, 
operate  independently.  Subsets  of  the  SoS  can  perform  SoS 
missions  in  the  absence  of  the  rest  of  the  system. 

Managerial 

Independence 

Constituent  Systems  of  the  SoS  are  successfully  developed 
and  integrated  independently  outside  the  SoS. 

Evolutionary 

Development 

The  constituent  Systems  and  SoS  are  built  over  many  years 
and  capabilities  are  added  incrementally. 

Emergent 

Behavior 

The  full  spectrum  of  layered  SoS  can  only  be  provided  by  all 
constituent  Systems  operating  together. 

Geographic 

Distribution 

Constituent  Systems  of  the  SoS  are  distributed  world-wide, 
may  be  mobile,  and  communicate  via  data  networks. 
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Problem 

Characteristic 

Simple 

Complex 

Your 

System 

Complexity  Rationale 

Number  of 
Elements 

Small 

Large 

? 

•  Internal  Social: 

Multiple  competing  Directorates,  Services/Program 
Offices,  Contractors,  UARCs/FFRDCs,  Warfighters 

•  Internal  Technical: 

Large  number  of  systems,  sub-systems,  etc. 

•  External  Social: 

Congress,  POTUS,  DOT&E,  Services,  Warfighters 

•  External  Technical: 

Other  services  necessary,  such  as  communications,  not 
under  system’s  control 

Interactions 

Few 

Many 

? 

•  Technical: 

The  SoS  interacts  between  Systems  and  externals 
through  numerous  links  and  interfaces 

•  Social: 

SoS  interactions  occur  across  multiple  Design, 
Development,  W&A,  Test,  Operational  teams  within 
Service,  Contractors,  Program  Offices 

Predetermined 

Attributes 

Yes 

No 

? 

•  Wide  variety  of  Systems 

•  Multiple  disparate  stakeholders 

•  SoS  organization  changes  -  over  time  based  on 
operational  SoS  and  Systems 

•  System  boundaries  difficult  to  define  -  especially  with 
regard  to  the  operator  being  a  part  of  the  system  or 
external  to  it. 
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Problem 

Characteristic 

Simple 

Complex 

Your 

System 

Complexity  Rationale 

Interaction 

Organization 

Highly 

Organized 

Loosely 

Organized 

? 

•  Many  activities  are  centrally  organized  but  executed 
locally  or  distributed. 

•  Systems,  Sub-systems,  Service,  Operators  are  differently 
organized. 

Laws  Governing 
Behavior 

Well  Defined 

Probabilistic 

? 

•  Degree  to  which  the  behavior  of  the  SoS  is  predictable  - 
emergent  behavior 

System  Evolution 
Over  Time 

Does  Not 
Evolve 

Evolves 

? 

•  Overlapping  complex  planned  SoS  evolution  at  System 
and  Sub-system  levels  over  different  time  frames. 

Subsystems 
Pursue  Own 
Goals 

No 

Yes 

(Purposeful) 

? 

•  Programs  and  services  are  driven  toward  meeting  the 
goals  of  their  own  systems 

•  Many  Systems  are  complex  in  their  own  rights  and  have 
missions  other  than  that  of  the  SoS. 

Systems  Affected 
by  Behavioral 
Influences 

No 

Yes 

? 

•  Different  service  paradigms  operating  in  “joint”  world 

•  Differences  between  R&D,  Developers,  Integrators, 
Testers,  Operators 

Predominantly 
Closed  or  Open 
to  the 

Environment 

Largely 

Closed 

Largely 

Open 

? 

•  Primary  purpose  of  the  system  is  to  address  external 
factors 

•  Subject  to  environment 

•  Depends  upon  external  services  (such  as  comms)  to 
accomplish  mission 
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Wicked  Problems  and  Social 

Complexity * 


1.  You  don’t  understand  the  problem  until  you  have  developed  a  solution. 

-  Every  SoS  “solution”  exposes  new/different  aspects  of  the  problem. 

-  The  problem  is  ill-structured  with  evolving  interrelated  issues  and  constraints. 

-  Different  stakeholders  have  different  viewpoints. 

2.  Wicked  problems  have  no  stopping  rule. 

-  Problem  solving  is  usually  limited  by  resources  rather  than  discovery  of  an  optimal 
solution. 

3.  Solutions  to  wicked  problems  are  not  right  or  wrong. 

-  There  is  no  optimal  solution,  but  “better  or  worse”,  “good  enough  or  not  good  enough.” 

4.  Every  wicked  problem  is  essentially  unique  and  novel. 

-  Many  factors  and  conditions  are  embedded  in  a  dynamic  socio-technical  context. 

5.  Every  solution  to  a  wicked  problem  is  a  “one-shot  operation.” 

-  You  cannot  build  a  SoS  just  to  see  how  it  works. 

-  But  you  can’t  learn  about  the  problem  without  trying  solutions. 

6.  Wicked  problems  have  no  given  alternative  solutions. 

-  There  may  be  no  solutions,  or  there  may  be  many  solutions. 

-  Creativity  in  selecting  solutions  and  judging  which  are  valid  is  critical. 
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Working  Group  3: 

Operations  Analysis  for  Systems  of  System  within  a 

Networked  C2  Context 


COMMAND  AND  CONTROL 
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•  A  case  study  of  a  civilian  command  and  control  system: 

-  Briefly,  the  concept  of  operations  for  the  ITS  is  to  Detect,  Surveil, 
Monitor,  and  Assess  arterial  and  freeway  roadways  for  traffic 
conditions,  road  conditions,  and  incidents.  This  information  is  used  to 
provide  control  of  the  roadway  such  as  changing  road  rules  (speed, 
direction,  etc.)  and  provide  information  to  other  system  elements.  In 
addition  the  ITS  provides  and  maintains  an  electronic  fee  collection 
system  for  road  and  transit  usage  and  provides  timely  and  useful 
information  to  travelers.  Additionally  the  ITS  manages  a  transit 
system,  manages  any  roadway  incidents  (by  changing  road  rules  and 
dispatching  resources  as  needed)  and  links  to  the  EMS  system  to 
respond  to  emergency  situations.  The  ITS  performs  some  of  its 
functions  autonomously  and  some  under  operator  control.  The  ITS 
links  to  other  information  systems  and  communications  networks  to 
receive  and  distribute  pertinent  information. 
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•  System  Diagrams 

-  Boundary  Diagram  for  the  Denver-Boulder  ITS. 

-  UML  Class  Diagrams  for  the  Denver-Boulder  ITS  and  represent  the  most 
current  information  from  2006  [ref.  8].  High  level  interfaces  are  only  shown  in 
first  for  clarity.  Second  figure  illustrates  the  class  structure  with  attributes 
and  operations  of  the  various  classes.  It  is  recognized  that  many  similar 
attributes  and  operations  appear  across  the  classes  which  could  lead  to  further 
refinement  producing  common  use  sub-classes.  Disseminatelnfo  is  an  example 
where  a  common  method  using  common  message  sets  could  be  applied. 

•  System  MOEs  and  MOPs 

-  The  table  outlines  the  ITS  MOEs  and  MOPs.  The  ITS  System  MOEs  are  both 
quantitative  (e.g.,  miles/%  of  roadway),  and  qualitative  (e.g.,  is  a  capability 
present).  The  ITS  MOPs  include  “ilities”  for  all  components  such  as,  Mean- 
Time-To-Failure,  Mean-Time-To-Repair,  Reliability,  Availability, 
Maintainability,  etc.  so  these  are  not  included  in  the  Table. 
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Define  System 
Boundaries 

in  terms  of: 

•  Entities 

•  Relationships 

•  Permeability 

•  Interdependence 

•  Hierarchy 

•  Relationship  to  External 
Factors  (Environment, 
Operator,  etc.) 


EMERGENCIES/ 
Exogenous  Events 


SPECIAL  EVENTS/ 
TOURIST  INFORMATION 


£ 

Operator 

* 

Passenger 

ft 

Payload 

ft 

1or2  Way 
Comm  Device(s) 


REGULATOR 

INSPECTOR 


1or2  WAY  COMM 
NETWORK(S) 

-  Radio/TV 

-  Wireless,  Cell 

-  Internet,  etc. 


OPERATOR 

MAINTAINER 

DEVELOPER 

VENDOR 


ENVIRONMENT 

-  Terrain 

-  Weather 

-  Road  Conditions 


STANDARDS 

LAWS 

REGULATIONS 

POLICY 


INFRASTUCTURE 

-  Roadways 

-  Parking  Lots 

-  Traffic  Controls 
-Etc. 
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High-Level  Class 
Interfaces 
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Denver-Boulder 

ITS 


— — 

Electronic  Payment  System 


Paymentlnformation 


CollectPayment(Toll,  Parking, 
Multi-use,  TransitFare) 


Roadway  Mgmt  System 


RoadwaySituatiuonAwareness 

RoadwayControl 


MonitorRoadway 
Control  Roadway 
Disseminatelnfo 


1 

Transit  Mgmt  System 


SurveillanceData, 
EmployeeRecords,  RideSharelnfo, 
FleetData,  InformationalMessages 

Surveil(ln-Vehicle,  Facility) 
CredentialEmployee(EmployeelD) 
DisableRemoteSystem(systemlD) 
RideShare(Share/Match,  Dynamic 
Route/Schedule, 
ServiceCoordination) 
ManageFleet(AVL/CAD,  Planning, 
Maintenance) 

Dissmeninatelnfo(ln-Terminal/ 
Wayside,  Internet/Wireless/Phone, 

In-Vehicle) 


n 


Traveler  Information  System 

Type= Pre-Trip,  En-Route, 

Pre-Triplnfo 

Event,  Tourism 

En-Routelnfo 

Mode=lnternet/Wireless,  511, 

Events 

Other  Telephone,  TV/Radio, 

Tourism 

Kiosk,  IVS,  TravelService, 

£ 

AdvancedParking, 

Disseminatelnfo(Type,  Mode) 

ElectronicPayment 

i 

Incident  Mgmt  System 


Detection-SurveillanceData, 

Responselnfo 

Clear-RecoveryData 

InformationalMessages 

Detect&Surveil(Detector,  Imaging, 
TravelerReported,WirelessE91 1 , 
Mayday/ACN,  CallBox) 
MobilizeResponse(AVL/CAD, 
Route,  ServicePatrol) 
Dissemenatelnfo(DMS,  HAR) 
Clear&Recover(Video, 
TempTrafficControl,  Investigation) 


Emergency  Mgmt  System 


HAZMATData, 

EmergencyMedicalData 

Response-RecoveryData 

ManageHazardousMaterials(T  rack, 
Detection,  DriverAuthentication, 
PlannedRoute) 

ProvideEmergencyMedicalService( 
Telemedicine,  AdvancedACN) 
Respond&Recover(EarlyWarning, 
RepsonseMgmt,  Evacuation,  Re- 
Entry,  EmergencyTravelerlnfo) 


Arterial  Mgmt  System 


RoadID,  Intersection  ID,  ParkingID, 

TrafficControlInfo 

TrafficStatistics,  Situationlnfo, 

LaneStatus,ParkStatus, 

InformationalMessages, 

LawEnforcementData 


Surveil(Traffic,  Intersection) 
ControlTraffic(Signal,  Emergency, 

Speed,  Bicycle,  Pedestrian, 
SpecialEvent) 

ManageLane(HOV,  Reverse,  Price, 
Control,  SpeedLimit, 
EmergencyEvacuation) 
ManageParking(lnfo,  CollectedData) 
Dissemenatelnfo(DMS,  HAR,  In-Vehicle) 
EnforceLaw(Speed,  Stop/Yield) 


Freeway  Mgmt  System 


HwylD,  RampID, 
TrafficControlInfo, 
TrafficStatistics,  Situationlnfo, 
LaneStatus,  Eventlnfo, 
InformationalMessages, 
LawEnforcementData 


Surveil(Traffic,  Infrastructure) 
ControlRamp(Metering,  RampCIosure, 
PriorityAccess) 

ManageLane(HOV,  Reverse,  Price, 
Control,  SpeedLimit, 
EmergencyEvacuation) 
ManageSpecialEvent(Type,  TMS,  TMC) 
Dissemenatelnfo(DMS,  HAR,  In-Vehicle) 
EnforceLaw(Speed,  HOV, 
RampMetering) 


ACN  -  Advanced  Crash  Notification  K 

AVL  -  Automated  Vehicle  Location 

CAD  -  Computer  Aided  Dispatch 

DMS  -  Dynamic  Message  Signs 

HAR  -  Highway  Advisory  Radio 

HOV  -  High  Occupancy  Vehicle 

IVS  -  In-Vehicle  System 

TMC  -  Transportation  Management  Center 

TMS  -  Transportation  Management  System 

Light  blue  indicates  a  planned  feature  for 
which  technology  is  not  yet  present 
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Component 

Area 

System  MOE  Examples 

Component  MOP  Examples 

Roadway 

Management 

Survey  &  Detect 

Percentage  of  roads,  percentage  of  intersections  &  ramps 
with  electronic  data  collection,  video,  audio. 

Audio  detection  frequency  (Hz),  sensitivity  (dB).  Video 
band,  frame  size  (pixels),  data  rate 

Control 

Percentage  of  signalized  intersections  with:  signal  priority 
for  transit/emergency  vehicles,  signal  controls,  variable 
speed  limit,  ped/bike  signals 

Percentage  of  freeway  ramps  with:  ramp  metering,  ramp 
closure,  priority  access  for  transit  vehicles 

Control  data  rate  &  bandwidth,  control  delay.  Control 
message  latency,  accuracy. 

Lane 

Percentage  of  roads  with:  HOV,  reversible  lanes,  lane 
control  (e.g.  closure),  variable  speed  limit,  support 
emergency  evacuation 

Time  to  reverse/close  lane, 

Parking 

Parking  availability  monitored  and  distributed? 

Availability  timeliness,  accuracy. 

Special  Event 

Portable  transportation  management  systems  deployed? 

Transmission  range  to  ITS  node,  date  rate,  latency 

Info 

Number  of  DMS  deployed,  percentage  of  road  miles 
covered  by  HAR.  Are  IVS  used? 

DMS  range  to  data  link  node.  Data  rate,  accuracy.  IT 
requirements,  compute  power,  data  storage,  etc. 

Enforcement 

Automated  speed  enforcement  employed?  Percentage 
of  signalized  intersections  using  photo  enforcement 

Speed,  identification  accuracy.  Photo  enforcement 
detection  accuracy. 

Incident 

Management 

Survey  &  Detect 

Number  of  magnetic/acoustic  sensors  per  centerline 
mile.  Percentage  of  road  miles  with  video.  Wireless 
911/ACN  systems  deployed?  Call  boxes  per  road  mile.  Is 
traveler  reported  information  collected  and  used? 

Audio  detection  frequency  (Hz),  sensitivity  (dB).  Video 
band,  frame  size  (pixels),  data  rate 

Mobilize  & 
Respond 

EMS  use  AVL/CAD  to  locate  reported  incidents?  Is  there 
a  response  routing  system?  Patrols  per  road  mile. 

AVL  location  accuracy,  timeliness.  CAD  dispatch  delay, 
accuracy. 

Clear  &  Recover 

Automated  systems  used  to  clear  incident?  Video  to 
support  data  collection?  Temporary  traffic  control 
devices  used? 

Clearance  time  required.  Video  band,  frame  size  (pixels), 
data  rate 

Info 

Number  of  DMS  systems  deployed,  number  and 
percentage  of  centerline  miles  covered  by  HAR. 

DMS  transmit  range  to  ITS  node,  date  rate,  latency.  HAR 
frequency,  timeliness,  accuracy  of  broadcast.  IT 
requirements,  compute  power,  data  storage,  etc. 
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Denver-Boulder  ITS  MOEs  and  MOPs 

(2  of  2) 


c 

z 

o 

OSSIFIED 

Component 

Area 

System  MOE  Examples 

Component  MOP  Examples 

Transit 

Management 

Safety  &  Security 

Percentage  of  transit/EMS  vehicles  with  AVL/CAD,  automated 
maintenance  monitoring? 

AVL  location  accuracy,  timeliness.  CAD  dispatch 
delay,  accuracy.  Maintenance  accuracy, 
timeliness. 

Transport  Demand 

Ride  sharing/carpool  matching,  service  coordination,  dynamic 
routing/scheduling? 

Database  size,  time  to  perform  match,  personal 
data  security. 

Fleet 

Percentage  of  transit/EMS  vehicles,  transit  stations  with 
audio/video  surveillance  systems. 

Audio  detection  frequency  (Hz),  sensitivity  (d B). 
Video  band,  frame  size  (pixels),  data  rate.  Range. 

Info 

Percentage  of  transit/  EMS  vehicles  using  IVS  .  Percentage  of  transit 
stations  using  in-terminal  displays.  Internet,  wireless,  phone  to 
distribute  information? 

IT  requirements,  compute  power,  data  storage, 
etc. 

Emergency 

Management 

HAZMAT 

Does  the  system  track  HAZMAT  and  authenticate  drivers,  use 
HAZMAT  detectors,  provide  route  planning? 

HAZMAT  track  quality  (pos,  vel),  accuracy, 
timeliness,  latency. 

EMS 

Does  the  system  use  ACN  data,  are  ambulances  telemedicine 
capable? 

Crash  notification  timeliness,  accuracy. 

Response  & 
Recovery 

Early  warning  capability,  AVL/CAD  to  locate  incidents,  coordination 
of  evacuation  and  traffic  management,  emergency  traveler 
information? 

Probability  of  early  warning,  accuracy, 
timeliness.  AVL  location  accuracy,  timeliness. 
CAD  dispatch  delay,  accuracy. 

Electronic 

Payment 

Toll 

Percentage  of  toll  stations  with  electronic  collection  systems  (e.g., 
smartcard). 

Toll  accuracy,  reader  accuracy 

Transit  Fare 

Percentage  of  transit  vehicles  with  electronic  collection  systems 
(e.g.,  smartcard). 

Fare  accuracy,  reader  accuracy 

Parking 

Are  automated  parking  fee  payment  systems  deployed? 

Fee  accuracy,  reader  accuracy.  System 
resilience  (to  theft,  for  example). 

Multi-use 

Electronic  collection  systems  compatible  across  system? 

Multi-use  accuracy,  reader  accuracy 

Traveler  Info 

Pre-Trip  & 

En-Route 

Use  internet/wireless,  511,  other  telephone,  radio/TV,  kiosks  to 
distribute  pre-trip/en-route  info? 

Timeliness,  accuracy,  completeness  of  data. 

Tourism  &  Events 

Provide  traveler  service  information  (lodging,  points  of  interest, 
etc.),  parking  information? 

Timeliness,  accuracy,  completeness  of  tourist, 
parking  data. 
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•  A  case  study  of  a  civilian  production  system: 

•  System  Diagrams 

-  The  first  figure  shows  a  simple  class  diagram  for  the  Ford  Production 
system  (FPS).  The  class  diagram  does  not  explicitly  show 
dependencies  between  production  line,  design,  and  management; 
however,  each  class  can  affect  the  other  classes.  For  example,  design 
change  may  cause  the  production  line  to  change,  or  a  production  line 
limitation  may  constrain  new  design  possibilities.  Consider  this  class 
diagram  to  be  a  stable  intermediate  state.  The  second  figure  shows 
the  simplified  functional  flow  for  the  FPS. 

•  System  Risk/Opportunity 

-  The  table  outlines  areas  where  FPS  and  Toyota  Production  System 
(TPS)  address  architectural  risk 
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Ford  Production  System 
Functional  Diagram  (IDEFO) 


Build  Team 
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.  FPS  Discussion:  Production  Line 


&  Modularity 


From  the  standpoint  of  the  class  diagram  and  functional  flow  both  the  FPS  and  TPS  have  a  lot  in 
common.  This  comes  from  the  fact  that  the  TPS  is  based  on  the  mass  production  concept  but 
overlaid  with  lean  manufacturing  concepts.  The  basic  functions  and  activities  are  similar;  however, 
how  these  operations  are  carried  out  is  somewhat  different  as  are  the  interactions  between  classes. 

It  may  require  behavior  diagrams  to  identify  the  differences.  Ford  focused  on  a  machine  view  of 
the  system  where  the  human  served  the  production  line.  The  processes  were  established  to 
eliminate  thought  and  skill  from  the  process  as  much  as  possible.  Toyota  takes  a  more  holistic  view 
of  the  place  of  the  human  within  the  production  process  and  recognizes  that  the  quality, 
productivity,  process  improvement,  and  morale  are  all  increased  when  the  needs  of  the  worker  are 
also  accounted  for.  TPS  adapted  the  FPS,  which  was  highly  skewed  toward  technical  aspects  of 
mass  production,  and  included  a  greater  focus  on  the  social  aspects  of  mass  production.  TPS  was 
driven  as  much  by  the  economies  (through  elimination  of  waste  -  muda,  mura,  muri)  that  could  be 
achieved  by  that  focus  as  the  social  good  that  the  more  holistic  view  provided. 


Ford  applied  the  concept  of  modularity  in  both  an  architectural  and  system  fashion.  The  major 
architectural  approach  to  modularity  was  to  break  down  the  production  of  the  car  into  discrete, 
sequential  steps  and  to  implement  the  moving  assembly  line  by  which  the  car  moved  past  various 
workstations  in  turn  for  assembly  from  start  to  finish.  From  a  system  standpoint  the  production 
line  was  designed  so  that  multiple  shifts  could  operate  independent  from  each  other  and  produce  the 
same  car  through  the  same  processes  in  the  same  amount  of  time. 
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Ford  option  analysis  focused  on  wringing  every  possibly  economy  out  of  the  mass  production  model, 
making  the  assembly  line  as  efficient  and  cost  effective  as  possible.  Ford’s  main  approach  seemed  to  be  to 
optimize  architecture  for  least  cost  at  any  particular  point  while  Toyota  addressed  elimination  of  sources  of 
waste,  thereby  reducing  cost,  across  the  entire  enterprise.  TPS  may  be  more  flexible  to  sub-optimize  any 
part  of  the  process  so  that  the  entire  process  is  optimized.  Toyota  focused  on  economizing  the  entire 
enterprise  from  supplier  to  producer  to  consumer. 

An  example  where  options  analysis  applied  to  modularity  may  have  improved  the  system  has  to  do  with 
Ford’s  concept  to  simplify  individual  tasks  as  much  as  possible  so  that  a  worker  stayed  in  one  place  doing 
one  task  at  a  particular  station  on  the  assembly  line.  One  downside  of  the  Ford  approach  was  social 
alienation  of  the  worker,  boredom,  and  repetitive  stress  injuries.  This  increased  the  risk  of  defects  through 
inattention  and  only  being  able  to  recognize  defects  in  a  very  limited  scope.  A  real  options  approach  could 
have  balanced  cost,  worker  satisfaction,  work  modularity,  and  quality/safety.  The  completion  of  a 
component  of  the  car  may  require  the  use  of  multiple  tools  to  assemble  all  parts  and  sub-assemblies.  In  the 
FPS  system  one  operator  mans  each  tool  and  the  component  is  passed  from  operator  to  operator  until  it  is 
assembled.  In  the  TPS  system  a  single  operator  may  be  responsible  to  carry  the  component  from  station  to 
station  and  perform  the  assembly  task  at  each  station.  The  FPS  systems  requires  a  number  of  operators 
based  on  the  number  of  tooling  stations  so  an  individual  operator  may  be  idle  at  times  depending  on  the 
rate  of  production.  In  the  TPS  system  the  number  of  operators  can  be  varied  to  reduce  idle  time  based  on 
the  production  rate.  In  the  TPS  system  the  operator  must  be  more  skilled  because  he  must  learn  multiple 
tasks.  This  has  an  advantage  in  that  it  should  generally  improve  worker  satisfaction  through  variety  and 
increase  quality,  thereby  reducing  enterprise  cost,  through  ownership  and  better  understanding  of  entire 
component.  The  downside  as  compared  to  FPS  is  that  more  skilled  labor  is  required,  and  FPS  seeks  to 
eliminate  the  need  for  skill  and  therefore  reduce  cost. 
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Risk/Opportunit 
y  Source 

Addressed  by  Architecture?  (Yes,  No,  Partial) 

FPS 

Comment 

TPS 

Comment 

Overproduction 

Part 

Ford  production  “pushes”  through 
the  line  so  requiring  more 
capital/storage  and  higher  risk  of 
overproduction  if  market  is 
misjudged. 

Yes 

Greater  focus  on  just  in  time 
techniques.  Less  storage/capital 
outlay  required  (Muda) 

Inventory 

Control 

Part 

Yes 

Unnecessary 

Transportation 

Yes 

Combined  material  processing, 
manufacturing,  and  assembly  at 
plant 

Yes 

Reduce  risk  of  damage,  loss,  or 
delays  to  product(Muda) 

Motion 

(worker/equipm 

ent) 

Yes 

Introduced  standardized  parts  and 
processes. 

Yes 

Reduce  damage  and  wear,  increase 
safety  (Muda) 

Defects 

Part 

Product  standardized  reduces 
defects,  but  may  be  offset  by 
unskilled  labor 

Yes 

Reduced  defects  =  reduced  cost  of 
rework/delays  (Muda) 

Overprocessing 

Part 

Transformed  craft  production  to 
mass  production 

Yes 

Use  tools  only  as  precise,  complex, 
or  expensive  as  to  meet  customer 
need  to  reduce  cost  (Muda) 

Waiting 

Part 

Moving  assembly  line 

Yes 

Reduced  waiting  =  reduced  risk  of 
damage,  loss  and  less  storage 
required  (Muda) 

Latent  Skill 

No 

Workers  had  one  job  at  one  position; 
reduced  flexibility/  adaptability, 
affects  defect  rate  and  morale 
through  boredom 

Part 

Capitalize  on  employees  others  skills 
to  improve  their  performance  and 
processes.  Developing  people  adds 
value.  (Muda) 
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Risk/Opportunit 
y  Source 

Addressed  by  Architecture?  (Yes,  No,  Partial) 

FPS 

Comment 

TPS 

Comment 

Production 

Leveling 

Part 

Improved  processes  enabled  shorter 
work  hours  and  expanded  work  from 

2  to  3  shifts.  Most  effective  when 
assembly  line  operating  at  full 
capacity  (push). 

Part 

Frequent  deliveries  to  customer  helps 
eliminate  overproduction  (pull). 

Merge  sub-processes  under  single 
worker.  Suboptimal  for  processes 
requiring  long  lead  times  (Mura) 

Work  Flow 
Optimization 

Yes 

Standardized  processes, 
interchangeability 

Yes 

Improved  quality/productivity, 
reduced  cost,  better  morale  (Muri) 

Repeatable 

Processes, 

Machine 

Processes 

Yes 

Decompose  complex  tasks  w/  special 
purpose  tools.  Improved 
quality/productivity,  reduced  cost, 
adaptable  flexibility  (Fordism) 

Yes 

Reasonable 
Process  Time 

Yes 

Optimized  processes  to  reduce 
production  time 

Yes 

Producing 
goods  that  do 
not  meet 

customer 

demands  or 
specifications 

Part 

Model  T  focused  on  cost  reduction 
through  mass  productions  (“push” 
model).  Later  displaced  by  complex 
mixture  of  engineering,  production, 
and  marketing  which  Ford  had  to 
catch  up  to. 

Yes 

TPS  organized  manufacturing  & 
logistics,  but  included  interactions 
with  suppliers  and  customers. 
Everyone  in  process  is  considered  as 
a  customer  and  supplier  (“pull” 
model). 

Product 

Standardization 

Yes 

Standard  parts  and  processes; 
improved  quality  and  reduced 
assembly  time  (Fordism) 

Yes 

Standard  parts  and  processes; 
improved  quality  and  reduced 
assembly  time  (copied  from  Fordism) 

Elimination  of 
skilled  labor 

Yes 

Direct  production  using  unskilled 
labor  is  less  expensive  but  can 
increase  defect  rate,  for  example 

No 

Developing  people  adds  value. 

(Muda) 
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SOS/Complex  System  definition 

C2  definition  (I  would  assume  it  could  include  C2,  C3,  CIS, 
C3I,  C4I,C4ISR,  C2BM,  C2BMC,  etc) 

Formulating  MOEs/MOPs  (esp.  in  context  of  C2) 

As  OSI-like  framework  for  C2  networks  (to  help  define  the 
types  of  MOEs  pertaining  to  different  views  of  the  system) 

Network  as  a  Cloud 

Mission  definition,  Commander’s  Intent  and  System/Problem 
context 

Quantitative  &  Qualitative  assessment  of  the  system 
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